Skip to content

Fix Flight LargeList schema encoding - #10298

Open
yinli-systems wants to merge 4 commits into
apache:mainfrom
yinli-systems:kevin/fix-flight-large-list-schema-10291
Open

Fix Flight LargeList schema encoding#10298
yinli-systems wants to merge 4 commits into
apache:mainfrom
yinli-systems:kevin/fix-flight-large-list-schema-10291

Conversation

@yinli-systems

@yinli-systems yinli-systems commented Jul 7, 2026

Copy link
Copy Markdown

Which issue does this PR close?

Rationale for this change

prepare_field_for_flight rebuilds DataType::LargeList with Field::new_list, silently changing the 64-bit offset type into a regular List in the Flight schema. The adjacent ListView / LargeListView path already preserves its large variant.

What changes are included in this PR?

  • Use Field::new_large_list for DataType::LargeList.
  • Exercise a real FlightDataEncoder to FlightDataDecoder IPC round trip covering List, nested LargeList<List>, FixedSizeList, parent and child nullability, and schema/field/child metadata.

Are these changes tested?

Yes, locally with Rust 1.96.1 on macOS:

  • Fault injection with the old new_list implementation: the round-trip regression test fails because decoded LargeList becomes List.
  • cargo test -p arrow-flight test_list_schema_round_trip
  • cargo test -p arrow-flight --lib (42 passed)
  • cargo test -p arrow-flight (unit, client, encode/decode, and doc tests passed)
  • cargo clippy -p arrow-flight --all-targets --all-features -- -D warnings
  • cargo fmt --all -- --check
  • git diff --check

I used AI assistance to inspect the nearby Flight schema-preparation paths and help generate the focused round-trip regression, then reviewed the resulting one-line production fix and validated it with the commands above.

Are there any user-facing changes?

Yes. Flight-encoded schemas now preserve LargeList instead of exposing it as List after schema preparation.

@github-actions github-actions Bot added arrow Changes to the arrow crate arrow-flight Changes to the arrow-flight crate labels Jul 7, 2026
@yinli-systems
yinli-systems marked this pull request as ready for review August 1, 2026 03:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

arrow Changes to the arrow crate arrow-flight Changes to the arrow-flight crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Schema for LargeList in Flight data encoder

1 participant